Nonsmooth cryptanalysis, with an application to the stream cipher MICKEY

A new approach to the cryptanalysis of symmetric algorithms based on nonsmooth optimisation is presented. We develop this technique as a novel way of dealing with nonlinearity over F2 by modeling the equations corresponding to the algorithm as a continuous optimisation problem that avoids terms of higher degree. The resulting problems are not continuously differentiable, but can be approached with techniques from nonsmooth analysis. Applied to the stream cipher MICKEY, which is part of the eSTREAM final portfolio, this method can solve instances corresponding to the full cipher, although with time complexity greater than brute force. Finally, we compare this approach to classical pseudo-Boolean programming.